chore(native): declare PyContextVar_* locally for limited-API builds (PROF-15850) by vlad-scherbich · Pull Request #19903 · DataDog/dd-trace-py

vlad-scherbich · 2026-08-27T19:34:13Z

Description

Delta vs #19861.

src/native/contextvar.rs declares PyContextVar_New / _Get / _Set locally and calls those instead of pyo3::ffi. pyo3-ffi omits them from limited-API bindings.

If this merges and nothing else new lands: limited-API contextvar builds compile. No publish, testrunner, riot, or requires-python change.

Testing

Native module compiles with the local declarations.

Risks

None. Declarations do not change runtime behavior on 3.9–3.14.

Additional Notes

No release note: internal compile fix, changelog/no-changelog.

Base: #19861. Next: #19907.

datadog-prod-us1-4 · 2026-08-27T19:35:05Z

✨ Unblock PR with BitsAI

⚠️ Warnings

❌ Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 5 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-py | core/internal 2/6 — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-py | rust ci — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-py | core/internal 1/6

View more details · View in GitLab

View all 5 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 792f843 | Docs | View more details | Give us feedback!}

pr-commenter · 2026-08-27T19:58:00Z

Benchmarks

Benchmark execution time: 2026-08-30 18:56:06

Comparing candidate commit 72d7f7a in PR branch vlad/py315-pyo3-contextvar with baseline commit 23f857d in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 6 performance regressions! Performance is the same for 575 metrics, 10 unstable metrics, 2 known flaky benchmarks, 16 flaky benchmarks without significant changes.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:httppropagationinject-ids_only

🟥 execution_time [+2.422µs; +2.576µs] or [+14.307%; +15.217%]

scenario:iastaspects-title_aspect

🟥 execution_time [+74.300µs; +82.515µs] or [+27.636%; +30.692%]

scenario:iastaspectsospath-ospathbasename_aspect

🟥 execution_time [+117.660µs; +125.166µs] or [+28.586%; +30.410%]

scenario:iastaspectssplit-rsplit_aspect

🟥 execution_time [+25.301µs; +28.744µs] or [+17.525%; +19.910%]

scenario:telemetryaddmetric-1-count-metric-1-times

🟥 execution_time [+591.550ns; +631.068ns] or [+21.684%; +23.133%]

scenario:tracer-small

🟥 execution_time [+31.118µs; +34.344µs] or [+9.368%; +10.340%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:coreapiscenario-context_with_data_listeners

unstable execution_time [-750.899ns; +713.817ns] or [-6.862%; +6.523%]

scenario:coreapiscenario-core_dispatch_1_listener

unstable execution_time [-31.896ns; +34.340ns] or [-5.231%; +5.632%]

scenario:coreapiscenario-core_dispatch_50_listeners

unstable execution_time [-1665.418ns; +1613.020ns] or [-9.789%; +9.481%]

scenario:coreapiscenario-core_dispatch_exception_listeners

unstable execution_time [-1402.493ns; +1129.487ns] or [-10.602%; +8.539%]

scenario:coreapiscenario-core_dispatch_listeners

unstable execution_time [-325.854ns; +324.990ns] or [-8.868%; +8.844%]

scenario:coreapiscenario-core_dispatch_no_args_listeners

unstable execution_time [-256.519ns; +256.867ns] or [-8.738%; +8.750%]

scenario:coreapiscenario-core_dispatch_with_results_1_listener

unstable execution_time [-78.553ns; +71.135ns] or [-6.672%; +6.042%]

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

unstable execution_time [-4078.986ns; +3809.804ns] or [-10.073%; +9.408%]

scenario:coreapiscenario-core_dispatch_with_results_listeners

unstable execution_time [-778.217ns; +816.348ns] or [-9.579%; +10.049%]

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

unstable execution_time [-60.193µs; +57.834µs] or [-9.628%; +9.250%]

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

scenario:iastaspects-casefold_noaspect

🟥 execution_time [+62.610µs; +68.388µs] or [+25.435%; +27.782%]

scenario:span-start

🟥 execution_time [+1.074ms; +1.241ms] or [+7.513%; +8.686%]

Known flaky benchmarks without significant changes:

scenario:errortrackingflasksqli-baseline
scenario:flasksimple-iast-get
scenario:iastaspects-casefold_aspect
scenario:iastaspects-index_aspect
scenario:iastaspects-ljust_noaspect
scenario:iastaspects-lower_aspect
scenario:iastaspects-replace_aspect
scenario:iastaspects-swapcase_aspect
scenario:iastaspects-title_noaspect
scenario:iastaspects-translate_aspect
scenario:iastaspects-translate_noaspect
scenario:iastaspects-upper_noaspect
scenario:packagespackageforrootmodulemapping-cache_off
scenario:packagespackageforrootmodulemapping-cache_on
scenario:sethttpmeta-all-enabled
scenario:telemetryaddmetric-record-100-metrics

vlad-scherbich · 2026-08-27T20:58:25Z

Part of #19908

Copilot

Pull request overview

This PR fixes Rust native extension builds under Py_LIMITED_API (e.g., CPython 3.15 builds using PYO3_USE_ABI3_FORWARD_COMPATIBILITY) by locally declaring the PyContextVar_New, PyContextVar_Get, and PyContextVar_Set C-API entry points that are not exposed by pyo3-ffi in that configuration.

Changes:

Add local extern "C" declarations for PyContextVar_* functions in src/native/contextvar.rs.
Switch call sites from ffi::PyContextVar_* to the locally declared symbols so the module compiles with and without Py_LIMITED_API.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

cit-pr-commenter-54b7da · 2026-08-27T21:12:04Z

Codeowners resolved as

Resolved from the full PR diff against vlad/py315-manylinux-wheels using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

src/native/contextvar.rs                                                @DataDog/apm-core-python
src/native/crashtracker/crashtracker_runtime_stacks.rs                  @DataDog/profiling-python @DataDog/apm-core-python

cit-pr-commenter-54b7da · 2026-08-27T21:12:06Z

Circular import analysis

⚠️ Existing circular imports

There are 3 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.llmobs -> ddtrace.llmobs._evaluators -> ddtrace.llmobs._evaluators.format -> ddtrace.llmobs._experiment -> ddtrace.llmobs
ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector
ddtrace.appsec._asm_request_context -> ddtrace.appsec._iast._iast_request_context_base -> ddtrace.appsec._iast._iast_env -> ddtrace.appsec._iast.reporter -> ddtrace.appsec._exploit_prevention.stack_traces -> ddtrace.appsec._asm_request_context

cit-pr-commenter-54b7da · 2026-08-27T21:12:16Z

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 240 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 240 highest severity)

ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=135)
ddtrace.appsec._contrib.flask -×-> ddtrace.trace  (product:appsec -> product:tracing, score=133)
ddtrace.llmobs._integrations.openai -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=133)
ddtrace.llmobs._integrations.mcp -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=133)
ddtrace.internal.test_visibility.api -×-> ddtrace.trace  (product:ci_visibility -> product:tracing, score=133)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:
uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5e2e90539b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Track B now forks at #19903 (PyContextVar_*). DoE attach stays #19272.

…imited-API PyContextVar_New/_Get/_Set are absent from pyo3-ffi limited-API bindings. PyGILState_Check is not limited-API; on those builds skip the probe rather than acquire or guess from a crash signal handler.

vlad-scherbich · 2026-08-31T18:36:13Z

@gyuheon0h can you please take a look if this change makes sense here as part of the PR stack?

vlad-scherbich · 2026-08-31T18:37:12Z

@vlad-scherbich update summary, add an example crash this is fixing.

emmettbutler

Seeing an example of the failure that occurs without this change would be helpful. Do we build with multiple values of the FORWARD_COMPATIBILITY flag? Why is it necessary to support multiple values of that flag?

vlad-scherbich · 2026-08-31T18:40:14Z

@vlad-scherbich update summary, add an example crash this is fixing.

Track B now forks at #19903 (PyContextVar_*). DoE attach stays #19272.

vlad-scherbich mentioned this pull request Aug 27, 2026

[3.15] Native/contextvar: PyContextVar_* hidden under Py_LIMITED_API #19908

Open

vlad-scherbich added the changelog/no-changelog A changelog entry is not required for this PR. label Aug 27, 2026

vlad-scherbich marked this pull request as ready for review August 27, 2026 21:03

vlad-scherbich requested a review from a team as a code owner August 27, 2026 21:03

vlad-scherbich requested review from juanjux and a lite review from Copilot August 27, 2026 21:03

vlad-scherbich changed the title ~~chore(native): declare PyContextVar_* locally for limited-API builds~~ chore(native): declare PyContextVar_* locally for limited-API builds (PROF-15850) Aug 27, 2026

Copilot started reviewing on behalf of vlad-scherbich August 27, 2026 21:03 View session

Copilot AI reviewed Aug 27, 2026

View reviewed changes

Comment thread src/native/contextvar.rs Outdated

vlad-scherbich force-pushed the vlad/py315-pyo3-contextvar branch from 5e2e905 to 9c1783d Compare August 27, 2026 21:11

vlad-scherbich requested a review from a team as a code owner August 27, 2026 21:11

vlad-scherbich changed the base branch from main to vlad/no-cp315-to-pypi August 27, 2026 21:11

This was referenced Aug 27, 2026

ci(testrunner): rebuild 3.15-dev against current CPython #19905

Closed

ci: withhold cp315 wheels from PyPI and the prerelease index (PROF-15857) #19880

Open

chatgpt-codex-connector Bot reviewed Aug 27, 2026

View reviewed changes

Comment thread src/native/contextvar.rs

vlad-scherbich requested a lite review from Copilot August 27, 2026 22:37

Copilot started reviewing on behalf of vlad-scherbich August 27, 2026 22:38 View session

Copilot AI reviewed Aug 27, 2026

View reviewed changes

vlad-scherbich force-pushed the vlad/no-cp315-to-pypi branch from f668305 to def36ad Compare August 28, 2026 14:08

vlad-scherbich force-pushed the vlad/py315-pyo3-contextvar branch 2 times, most recently from f9ee924 to 12acbff Compare August 28, 2026 14:57

vlad-scherbich requested a review from a team as a code owner August 28, 2026 14:57

vlad-scherbich changed the base branch from vlad/no-cp315-to-pypi to vlad/py315-manylinux-wheels August 28, 2026 14:58

vlad-scherbich force-pushed the vlad/py315-pyo3-contextvar branch from 12acbff to 59047aa Compare August 28, 2026 15:51

vlad-scherbich requested review from a team as code owners August 28, 2026 15:51

vlad-scherbich requested review from KowalskiThomas and dubloom and removed request for a team August 28, 2026 15:51

vlad-scherbich changed the base branch from vlad/py315-manylinux-wheels to vlad/315-profiling-dev-tooling August 28, 2026 15:52

vlad-scherbich force-pushed the vlad/py315-pyo3-contextvar branch from 59047aa to bb9aaf9 Compare August 28, 2026 16:07

vlad-scherbich changed the base branch from vlad/315-profiling-dev-tooling to vlad/py315-manylinux-wheels August 28, 2026 16:07

vlad-scherbich force-pushed the vlad/py315-manylinux-wheels branch from 4362679 to c493409 Compare August 28, 2026 18:33

vlad-scherbich force-pushed the vlad/py315-pyo3-contextvar branch from bb9aaf9 to c2e9f4f Compare August 28, 2026 18:33

vlad-scherbich mentioned this pull request Aug 28, 2026

ci(testrunner): rebuild 3.15-dev and pin Cython<3.3 on the 3.15 cache (PROF-14439) #19907

Open

vlad-scherbich added a commit that referenced this pull request Aug 30, 2026

docs(profiling): restack map onto #19903

7165139

Track B now forks at #19903 (PyContextVar_*). DoE attach stays #19272.

vlad-scherbich mentioned this pull request Aug 30, 2026

chore(315): logging version warning and FunctionStore restore #19267

Open

vlad-scherbich added a commit that referenced this pull request Aug 30, 2026

docs(profiling): restack map onto #19903

5176e04

Track B now forks at #19903 (PyContextVar_*). DoE attach stays #19272.

vlad-scherbich force-pushed the vlad/py315-manylinux-wheels branch from cac61ca to 284bca6 Compare August 31, 2026 18:27

chore(native): declare PyContextVar_* locally and skip GIL probe on l…

792f843

…imited-API PyContextVar_New/_Get/_Set are absent from pyo3-ffi limited-API bindings. PyGILState_Check is not limited-API; on those builds skip the probe rather than acquire or guess from a crash signal handler.

vlad-scherbich force-pushed the vlad/py315-pyo3-contextvar branch from 72d7f7a to 792f843 Compare August 31, 2026 18:30

vlad-scherbich commented Aug 31, 2026

View reviewed changes

emmettbutler reviewed Aug 31, 2026

View reviewed changes

vlad-scherbich commented Aug 31, 2026

View reviewed changes

vlad-scherbich added a commit that referenced this pull request Aug 31, 2026

docs(profiling): restack map onto #19903

3930d00

Track B now forks at #19903 (PyContextVar_*). DoE attach stays #19272.

Uh oh!

Conversation

vlad-scherbich commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Risks

Additional Notes

Uh oh!

datadog-prod-us1-4 Bot commented Aug 27, 2026 • edited by datadog-datadog-us1-prod Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

pr-commenter Bot commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Explanation

More details about the CI and significant changes

scenario:httppropagationinject-ids_only

scenario:iastaspects-title_aspect

scenario:iastaspectsospath-ospathbasename_aspect

scenario:iastaspectssplit-rsplit_aspect

scenario:telemetryaddmetric-1-count-metric-1-times

scenario:tracer-small

Unstable benchmarks

scenario:coreapiscenario-context_with_data_listeners

scenario:coreapiscenario-core_dispatch_1_listener

scenario:coreapiscenario-core_dispatch_50_listeners

scenario:coreapiscenario-core_dispatch_exception_listeners

scenario:coreapiscenario-core_dispatch_listeners

scenario:coreapiscenario-core_dispatch_no_args_listeners

scenario:coreapiscenario-core_dispatch_with_results_1_listener

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

scenario:coreapiscenario-core_dispatch_with_results_listeners

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

Known flaky benchmarks

scenario:iastaspects-casefold_noaspect

scenario:span-start

Uh oh!

vlad-scherbich commented Aug 27, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

cit-pr-commenter-54b7da Bot commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codeowners resolved as

Uh oh!

cit-pr-commenter-54b7da Bot commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Circular import analysis

⚠️ Existing circular imports

Uh oh!

cit-pr-commenter-54b7da Bot commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependency direction analysis

⚠️ Existing dependency direction violations

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

vlad-scherbich Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

vlad-scherbich Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

emmettbutler left a comment

Choose a reason for hiding this comment

Uh oh!

vlad-scherbich Aug 31, 2026

vlad-scherbich commented Aug 27, 2026 •

edited

Loading

datadog-prod-us1-4 Bot commented Aug 27, 2026 •

edited by datadog-datadog-us1-prod Bot

Loading

pr-commenter Bot commented Aug 27, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Aug 27, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Aug 27, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Aug 27, 2026 •

edited

Loading